Supplementary Materials of Bridging the Ultimate Semantic Gap: A Semantic Search Engine for Internet Videos

In the multimodal search component, the LM-JM model (λ = 0.7) is used for ASR/OCR for the frequent-words in the event-kit description. BM25 is used for ASR [8] and OCR features for the event name query (1-3 words), where k1 = 1.2 and b = 0.75. Both the frequent-words query and the event name query are automatically generated without manual inspection. While parsing the frequent words in the event-kit description, the stop and template words are first removed, and words in the evidence section are counted three times. After parsing, the words with the frequency ≥ 3 are then used in the query. VSM-tf model is applied to all semantic concept features. In the SQG component, the exact word matching algorithm finds the concept name in the frequent event-kit words (frequency ≥ 3). The WordNet mapping uses the distance metrics in [10] as the default metric. We build an inverted index over the Wikipedia corpus (about 6 million articles), and use it to calculate the PMI mapping. A pre-trained word embedding trained on Wikipedia [6] is used to calculated the Word embedding mapping. In the PRF component, the SVM model is selected as the reranking model. The self-paced function used is the mixture weighting: